Stark: Fast and Scalable Strassen’s Matrix Multiplication Using Apache Spark

نویسندگان

چکیده

This article presents a new fast, highly scalable distributed matrix multiplication algorithm on Apache Spark, called Stark , based Strassen’s algorithm. Stark preserves seven multiplications scheme in environment and thus achieves asymptotically faster execution time. It creates recursion tree of computation where each level the corresponds to division combination blocks stored form Resilient Distributed Datasets (RDDs). processes divide combine step parallel memorises sub-matrices by intelligently tagging it. To best our knowledge, is first implementation distribute Spark platform. We also report detailed complexity analysis for proposed algorithm, taking into account communication costs. Experimental results suggest that outperforms existing implementations – Marlin MLLib high sizes ( $\geq 16384\times 16384$ ). Our experiments reveal optimal block size, which shown from theoretical analysis. show experimental running times match closely. has been experimentally exhibits strong scalability with increasing number executors.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Scalable SDE Filtering and Inference with Apache Spark

In this paper, we consider the problem of Bayesian filtering and inference for time series data modeled as noisy, discrete-time observations of a stochastic differential equation (SDE) with undetermined parameters. We develop a Metropolis algorithm to sample from the high-dimensional joint posterior density of all SDE parameters and state time series. Our approach relies on an innovative densit...

متن کامل

Fast matrix multiplication using coherent configurations

We introduce a relaxation of the notion of tensor rank, called s-rank, and show that upper bounds on the s-rank of the matrix multiplication tensor imply upper bounds on the ordinary rank. In particular, if the “s-rank exponent of matrix multiplication” equals 2, then ω = 2. This connection between the s-rank exponent and the ordinary exponent enables us to significantly generalize the group-th...

متن کامل

Real-time News Recommendations using Apache Spark

Recommending news articles is a challenging task due to the continuous changes in the set of available news articles and the contextdependent preferences of users. Traditional recommender approaches are optimized for analyzing static data sets. In news recommendation scenarios, characterized by continuous changes, high volume of messages, and tight time constraints, alternative approaches are n...

متن کامل

Fast Matrix Multiplication

We give an overview of the history of fast algorithms for matrix multiplication. Along the way, we look at some other fundamental problems in algebraic complexity like polynomial evaluation. This exposition is self-contained. To make it accessible to a broad audience, we only assume a minimal mathematical background: basic linear algebra, familiarity with polynomials in several variables over r...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: IEEE Transactions on Big Data

سال: 2022

ISSN: ['2372-2096', '2332-7790']

DOI: https://doi.org/10.1109/tbdata.2020.2977326